Bundle org.nuxeo.ecm.core.storage.binarymanager.s3

In bundle group org.nuxeo.ecm.core.storage.binarymanager

Documentation

README.md
nuxeo-core-binarymanager-s3

This addon implements a BinaryManager that stores binaries in a S3 bucket. For efficiency, a local disk cache (with limited size) is also used.

Be sure to protect your nuxeo.conf (readable only by the nuxeo user) as the file will have your AWS identifiers (unless you are using instance roles).

Mandatory Parameters
- nuxeo.core.binarymanager=org.nuxeo.ecm.blob.s3.S3BlobProvider
- nuxeo.s3storage.bucket : the name of the S3 bucket (unique across all of Amazon, find something original!)
- nuxeo.s3storage.awsid : your AWS_ACCESS_KEY_ID
- nuxeo.s3storage.awssecret : your AWS_SECRET_ACCESS_KEY
If the awsid and/or awssecret are not set, the addon will try to use temporary credentials from the instance role (if any).

Optional Parameters
- nuxeo.s3storage.region : the region code your S3 bucket will be placed in. For us-east-1 (the default), don't set this parameter For us-west-1 (Northern California), use us-west-1 For us-west-2 (Oregon), use us-west-2 For eu-west-1 (Ireland), use EU For ap-southeast-1 (Singapore), use ap-southeast-1 For ap-northeast-1 (Tokyo), use ap-northeast-1 For sa-east-1 (Sao Paulo), use sa-east-1
- nuxeo.s3storage.cachesize : size of the local cache (default is 100MB).
- nuxeo.s3storage.bucket_prefix : bucket prefix
- nuxeo.s3storage.pathstyleaccess : if true, configures the client to use path-style access for all requests (default is false)
Crypto Parameters

With S3, you have the option to store your data encrypted. Note that the local cache will NOT be encrypted.

The S3 binary manager can use a keystore containing a keypair, but there are a few caveats to be aware of :
- The Sun/Oracle JDK doesn't always allow the AES256 cipher which the AWS SDK uses internally. Depending on the US export restrictions for your country, you may be able to modify your JDK to use AES256 by installing the "Java Cryptography Extension Unlimited Strength Jurisdiction Policy Files". See this link to download the files and installation instructions.
- Don't forget to specify the key algorithm if you create your keypair with the "keytool" command, as this won't work with the default (DSA). The S3 Binary Manager has been tested with a keystore generated with this command :
```
keytool -genkeypair -keystore </path/to/keystore/file> -alias <key alias>
    -storepass <keystore password> -keypass <key password>
    -dname <key distinguished name> -keyalg RSA
```
With all that preceded in mind, here are the crypto options (they are all mandatory once you specify a keystore) :
- nuxeo.s3storage.crypt.keystore.file : the absolute path to the keystore file
- nuxeo.s3storage.crypt.keystore.password : the keystore password
- nuxeo.s3storage.crypt.key.alias = the key alias
- nuxeo.s3storage.crypt.key.password = the key password
Server-side Encryption

S3 allows you to use server-side encryption to encrypt data at rest with keys managed by S3 itself. To activate-this mode, use:
- nuxeo.s3storage.crypt.serverside = true
See the related AWS documentation for more.

If you want to use Server-Side Encryption with AWS KMS–Managed Keys, specify your key id with:
- nuxeo.s3storage.crypt.kms.key = your-key-id
See the related AWS documentation for more.

Enable CloudFront Direct Download

Please, read carefully the CloudFront documentation to understand how you bind a CloudFront distribution to a S3 bucket. After you created a CloudFront distribution domain bound to your S3 repository. Accessing your objects is not restricted per default.

You have to enable the restriction viewer access (Look at Serving Private Content throught CloudFront), then each distribution URL must be signed to access the target object.

You have to set the Query String Forwarding and Caching on all parameters (Look at Configuring CloudFront to Cache Based on Query String Parameters) in order to correctly forward Content Disposition parameters to get a response with a correct filename.

CloudFront Mandatory Parameters

S3 parameters (except nuxeo.core.binarymanager) are mandatory, additionally CloudFront requires some new ones:
- nuxeo.core.binarymanager=org.nuxeo.ecm.core.storage.sql.CloudFrontBinaryManager
- nuxeo.s3storage.cloudfront.privKey: the absolute path of the private key file (.pem or .der). Read: Creating CloudFront Key Pairs for Your Trusted Signers
- nuxeo.s3storage.cloudfront.privKeyId: the private key id.
- nuxeo.s3storage.cloudfront.distribDomain: the distribution domain name.
- nuxeo.s3storage.cloudfront.protocol: the prefered protocol (default HTTPS)
- nuxeo.s3storage.cloudfront.fix.encoding: Enable a workaround to fix an error on CloudFront side (default false)
Building
```
mvn clean install
```
Testing

Running the unit tests requires some environment variables and System properties to be set, otherwise they are skipped.

Environment Variables
```
  AWS_REGION=eu-west-3
  AWS_ACCESS_KEY_ID=******
  AWS_SECRET_ACCESS_KEY=******
  AWS_ROLE_ARN=arn:aws:iam::783725821734:role/nuxeo-s3directupload-role
```
System Properties
```
mvn test \
  -nsu \
  -Dnuxeo.test.s3storage.bucket=nuxeo-platform-unit-tests \
  -Dnuxeo.test.s3storage.transient.bucket=nuxeo-platform-unit-tests-transient \
  -Dnuxeo.test.s3storage.policy.bucket=nuxeo-platform-unit-tests-policy \
  -Dnuxeo.test.s3storage.bucket_prefix=BUCKET_PREFIX \
  -Dnuxeo.test.s3storage.provider.test.bucket_prefix=TEST_BLOB_PROVIDER_PREFIX \
  -Dnuxeo.test.s3storage.provider.other.bucket_prefix=OTHER_BLOB_PROVIDER_PREFIX
```
Deploying

Install the Amazon S3 Online Storage Marketplace Package. Or manually copy the built artifacts into $NUXEO_HOME/templates/custom/bundles/ and activate the "custom" template.

About Nuxeo

Nuxeo dramatically improves how content-based applications are built, managed and deployed, making customers more agile, innovative and successful. Nuxeo provides a next generation, enterprise ready platform for building traditional and cutting-edge content oriented applications. Combining a powerful application development environment with SaaS-based tools and a modular architecture, the Nuxeo Platform and Products provide clear business value to some of the most recognizable brands including Verizon, Electronic Arts, Sharp, FICO, the U.S. Navy, and Boeing. Nuxeo is headquartered in New York and Paris. More information is available at www.nuxeo.com.

Resolution Order

[126, 829]

The resolution order represents the order in which components have been resolved by the Nuxeo Runtime framework. This range represents the minimal and maximal orders for this bundle's components.
You can influence this order by adding "require" tags in the component declaration, to make sure it is resolved after another component. It will also impact the order in which contributions are registered on their target extension point (see "Registration Order" on contributions).

Components

Packages

amazon-s3-online-storage

Maven Artifact

File	nuxeo-core-binarymanager-s3-2023.9.10.jar
Group Id	org.nuxeo.ecm.core
Artifact Id	nuxeo-core-binarymanager-s3
Version	2023.9.10

Manifest

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven 3.6.1
Built-By: root
Build-Jdk: 17.0.10
Bundle-ManifestVersion: 2
Bundle-Version: 2023.9.10-t20240314-090558
Bundle-SymbolicName: org.nuxeo.ecm.core.storage.binarymanager.s3;singlet
 on:=true
Nuxeo-Component: OSGI-INF/s3binarymanager-managment-contrib.xml,OSGI-INF
 /s3web-request-controller-contrib.xml,OSGI-INF/configuration-properties
 .xml,OSGI-INF/bulk-config.xml

Exports

Json Export Default Json serialization
Json Graph Json dependency graph
Json Contribution Stats Json statistics for contributions
CSV Contribution Stats CSV statistics for contributions

Charts

Raw Data: Json Contribution Stats

Contributions by Code Type

Loading data

Contributions by Target Extension Point

Loading data

Contributions by Studio Source

Loading data